Differential Imitation Learning for Sequential Prediction Appendix A. A relation between AggreVaTeD with Natural Gradient and AggreVaTe with Weighted Majority

A.1. Weighted Majority in Discrete MDPs For notation simplicity, for each state s 2 S , we represent the policy ⇡(·|s) as a discrete probability vector ⇡s 2 (A). We also represent dt as a S-dimension probability vector from S-d simplex, consisting of dt (s), 8s 2 S . For each s, we use Qt (s) to denote the A-dimension vector consisting of the state-action cost-to-go Qt (s, a) for all a 2 A. With this notation, the loss function `n(⇡) from Eq. 1 can now be written as: `n(⇡) = 1